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Abstract — Whereas many results are known about thresholds 
for ensembles of low-density parity-check codes under message- 
passing iterative decoding, this is not the case for linear program- 
ming decoding. Towards closing this knowledge gap, this paper 
presents some bounds on the thresholds of low-density parity- 
check code ensembles under linear programming decoding. 

I. Introduction 

Message-passing iterative (MPI) decoding and linear pro- 
gramming (LP) decoding are very efficient methods to achieve 
excellent decoding performance of low-density parity-check 
(LDPC) codes on a variety of channels. While an enormous 
amount of work has been devoted to the understanding of 
LDPC codes under MPI decoding (see e.g. [1], [2], [3], [4], 
[5], [6] for results on thresholds), comparably few results 
on the performance of LDPC codes using the more recent 
LP decoding are known. In this paper we provide analytical 
bounds on thresholds of LP decoding; these bounds establish 
necessary conditions for the existence of LP decoding thresh- 
olds. 

We note that the existence of such thresholds is not clear a 
priori. In contrast to [7], where we discuss cases where we can 
guarantee a threshold under LP decoding, here we will show 
cases where an LP decoding threshold does not exist. E.g., the 
sequence of random codes where each entry in a parity-check 
matrix is drawn independently from a Bernoulli distribution 
with arbitrary nonzero parameter 9 does not possess an SNR- 
threshold (in the AWGNC case) or an e-threshold (in the 
BSC case). Also, there are ensembles of codes that do not 
show arbitrary low probability of decoding error for any SNR 
(AWGNC) or any positive e (BSC) even if the code rate is 
allowed to approach zero and the variable degree is allowed 
to grow exponentially fast in the block length. 

Because LP decoding and MPI decoding are essentially 
equivalent in the case of the binary erasure channel (BEC), 
well-known results about MPI decoding for the BEC (see 
e.g. [2], [3]) can be used for making statements about LP 
decoding thresholds and therefore we will not deal any further 
with the BEC case here. 

The paper is structured as follows. After having concluded 
this introduction with some remarks about our notation, we 
will review ML and LP decoding in Sec. |n] The main part 



of the paper will be Sec. |ITl] where we present so-called 0- 
neighborhood-based bounds on the LP decoding threshold. 
Finally, in Sec. HVI we will present so-called 2-neighborhood- 
based bounds. 

Let us fix some notation. We let R, R + , and R++ be the 
set of real numbers, the set of non-negative real numbers, and 
the set of positive real numbers, respectively. Moreover, we 
will use the canonical embedding of the set F2 = {0, 1} into 
R. The convex hull (see e.g [8]) of a set A C R™ is denoted 
by conv(_4). If A is a subset of FJ? then coxw(A) denotes 
the convex hull of the set A after A has been canonically 
embedded in W 1 . Similarly, the conic hull (see e.g [8]) of a 
set A C R n will be denoted by conic(^4) and if A is a subset 
of F2 then conic(„4) denotes the conic hull of the set A after 
A has been canonically embedded in R n . The i-th component 
of a vector x will be called [x]i or xi and the element in the 
j-th row and i-th column of a matrix A will be called [A]^. 

Let C be a binary linear code defined by a parity-check 
matrix H of size m by n. Based on H, we define the sets 
X^X(H) 4 {l,...,n}, J 4 J(H) 4 {l,...,m},lj = 
Jj(H) = {i e 1 I [H]^ = 1} for each j € J, and J t = 
Ji(H) = {j £ J I [H]^ = 1} for each i e 1. Moreover, 
for each j E J we define the codes Cj = Cj(H) = {x £ 
F£ I hjX T = (mod 2)}, where h., is the j-th row of H. Note 
that the code Cj is a code of length n where all positions not in 
lj are unconstrained. For simplicity of notation, we will never 
indicate the parity-check matrix as an argument of X, J, etc.; 
it will be clear from the context to what parity-check matrix 
we are referring to. Finally, by a family of codes we will mean 
a sequence of (deterministicly or randomly generated) codes 
where the block length goes to infinity. 

II. ML and LP Decoding 

Let us use the above-mentioned code C for data transmission 
over a binary-input discrete memoryless channel with input 
alphabet X = {0,1}, output alphabet y, and channel law 
Py\x{y\ x )- Because the channel is memoryless, /V|x(y| x ) = 
Hiex P Y\x{yi\%i), where X = (Xi,...,X n ), where Y = 
(Yi, . . . , Y n ), where the random variable Xj denotes the 
channel input at time index i, and where the random variable 
Yi denotes the channel output at time index i. Upon observing 
Y = y, the maximum-likelihood (ML) decoding rule decides 



for 



x(y) = argmaxP Y |x(y|x). 

Let the i-th log-likelihood ratio i E X, be the random 
variable 

with realization 7^ = 7i (?/;)■ Then, noting that 

logiV|x(y*|^) = -7i^i + logfV|x(^|0), 
ML decoding can also be written as 



x(y) 



arg min V^a^ 

iSX 



Because the cost function is linear, and a linear function attains 
its minimum at the extremal points of a convex set, this is 
essentially equivalent to 

x(y) = arg min V^x*. 

xEconv(C) 1 — ' 
ztEX 

Although this is a linear program, it can usually not be 
solved efficiently because its description complexity is usually 
exponential in the block length of the code. 

However, one might try to solve a relaxation of the 
above minimization problem. Noting that conv(C) C 
C\j£j conv(Cj) (which follows from the fact that C = 
C\j£jCj), Feldman, Wainwright, and Karger [9], [10] defined 
the linear programming decoder (LP decoder) to be given by 
the solution of the linear program 



w(y) = arg min V 7* 

weOjgy conv(Cj) f— ' 



(1) 



Because of its importance, we will abbreviate the set 
C\j e j conv(Cj) by V = V(SX) and call it the fundamental 
polytope [11], [12]. The fundamental polytope V can be 
expressed using inequalities as follows [9], [10], [11], [12] 



V = < w e : 



Vi £ X : < LOi < 1 and 



Vj e J, vx' c Xj 



E 



iS I' 



123 



odd : 

< - 1 



When analyzing the decoding performance of LP decoding 
of a binary linear code that is used for data transmission 
over a binary-input output-symmetric channel, we can without 
loss of generality assume that the all-zeros codeword was 
sent. (See also [9] and [10] that discuss this so-called "C- 
symmetry" property.) We observe that a necessary (but usually 
not sufficient) condition that one decides for the all-zeros 
codeword in Q is that 1 



E 

iex 



7^ > for all u G V \ {0}. 



(2) 



'Actually, without changing the content of the statement in |2j, we can 
replace uj eV\ {0} by a) e V . 



It can easily be seen that this condition is equivalent to the 
condition that 

^ for a11 u e conic(P) \ {0}. (3) 

iei 

The set conic("P), which is the conic hull of the fundamental 
polytope, is called the fundamental cone JC = /C(H). In terms 
of inequalities, K. can be written as [9], [10], [11], [12] 

Vi G 1 : < u>i and 
Vj G J, W G Xj : 



K 



u> G 



The condition in can then be stated as 



E 



jiUi > for all u G K \ {0}. 



(4) 



We will use the following definition for the block decoding 
error event under LP decoding: it is the complement of the 
event that the all-zeros vector is the unique solution in Q. 

III. O-Neighborhood-Based Bounds on the 
Threshold for Regular LDPC Codes 

We focus our attention on (u> co i, u> row )-regular LDPC codes, 
i.e. codes defined by parity-check matrices that have uniform 
column weight w co i and uniform row weight w Iow . For these 
type of codes, we present a technique to obtain bounds 
on the threshold under LP decoding which we will call 0- 
neighborhood-based bounds; the choice for this name will 
become clear later on. 

Assumption 1: In the following, we will always assume that 
the all-zeros codeword was sent, i.e. we will not explicitly 
write the conditioning on X = when making statements 
involving probabilities. □ 

Under this assumption, the log-likelihood ratios T\, . . . ,T n 
are i.i.d. random variables and so there is a random variable T 
such that Ti - T for all i G I. Moreover, let Q C (KU{±oo}) 
be the support of the pdf of V. 

Example 2: Let us discuss the random variable T for three 
channels: the binary-input additive white Gaussian channel 
(AWGNC), the binary symmetric channel (BSC), and the 
binary erasure channel (BEC). 

• AWGNC (with modulation map ^ +y/E^, 1 i-> —\/T% 
and where the added noise has variance a 2 ): Q = R 
and r is a continuous random variable that is normally 
distributed with mean 2E c /a 2 and variance AE c /a 2 . 

• BSC (with cross-over probability e): Q = {±G} and V 
is a discrete random variable that takes on the value G 
with probability 1 — e and the value — G with probability 
e, where G = log(i^). 

• BEC (with erasure probability e): Q = {0, +00} and T 
is a discrete random variable that takes on the value +00 
with probability 1 — e and the value with probability e. 

□ 



Definition 3: Let 



= + E r < 



and T r 



E 

r« <o 



be random variables with realizations 7 pos and 7 nog , respec- 
tively. Note that T pos > and r ncg > w.p. 1. □ 
Lemma 4: With probability one we have 



lim 



r 



pOS 



= +E[r|r>o] • Pr(r>o) = 



lim ^= 



E[r|r<o] ■ Pr(r<o) 



+ OC 

7Pr(7) d 7, 
7Pr(7) <h- 



Proof: Follows easily from the weak law of large numbers. □ 
Lemma 5: Consider a code with a (w co i, w TO w) -regular 
parity-check matrix. Let 7 G Q n . A necessary condition that 
the LP decoder decides in favor of the all-zeros codeword is 



7 P c 



> uv, 



1. 



7ncg 



Proof: We saw in @ that a necessary condition for LP 
decoding to decide in favor of the all-zeros codeword is that 
Tliei li^i — f° r all u> G /C \ {0}. Let us construct a vector 
uj G W 1 as follows: 

(5) 



1 



if 7j > 
if 7* < 



It can easily be seen that uj G /C(H). 2 We obtain the necessary 
condition 

1 



< ^ ^ iLOi 



= -1 ' 7nog 



■ll>r. 



1 



' Tpos J 



which is equivalent to the necessary condition in the lemma 
statement. □ 
Theorem 6: Consider a family of codes that have 
(u> C oi, Wrow) -regular parity-check matrices. In the limit n — > 
00, a necessary condition such that the LP decoder decides in 
favor of the all-zeros codeword with probability one is 

E[r 1 r>o] Pr(r>o) 



E[r 1 r<o] Pr(r<o) 

or, equivalently, 



> w r , 



1, 



> Wr- 



/_oo7Pr(7) d7 
Proof: This follows upon combining Lemmas |4] and [5] □ 
Corollary 7: Consider the setup of Th.|6] If the memoryless 
channel is a BSC with cross-over probability e then the 
necessary condition in Th. [6] reads 

1 

£ < • 

^row 

Proof: For a BSC with cross-over probability e we obtain 

E[r l r>o] • Pr(r>0) = +G ■ (1 - e), 
- E[r 1 r<o] • Pr(r<0) = -(-G) • s = +G ■ e 

where G is defined as in Ex. |2 Therefore, the condition in 
Th. |6]is that > w row - 1, i.e. e < □ 

2 The vector tti can be seen as a generalization of the so-called canonical 
completion [11], [12], however instead of assigning values according to the 
graph distance with respect to a single node, we assign values according to 
the graph distance with respect to the set of nodes where ji is negative. 
Moreover, we assign only the values 1 and l/(ui r ow — 1). 
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Fig. 1. The solid line shows the capacity Cbsc(s) °f a BSC 
as a function of the cross-over probability e. The circles have the 
following meaning: the circle with label (iu co i,«i r0 w) shows the 
point (ii(«« co i,-i«row),£uB(««col, u 'row)), where R(w co i , w row ) and 
^UB^coli "row) are the designed rate and the threshold upper bound from 
Cor. [7] respectively, for (iu co i, io row ) -regular LDPC codes. 



Example 8: Fig.^tries to capture some of the implications 
of Th. [6]/ Cor. The circles in the plot that are to the left of 
the capacity curve yield non-trivial upper bounds on the error 
correction capability of LP decoding. (Note that the designed 
rate of a (w co i, w row )-regular LDPC code is 1 — and that 
the actual rate is lower bounded by this quantity.) □ 

Example 9: It is a surprising fact that the bounds in Th. [6] 
and Cor. do not depend on the variable degree w co \ at all. 
In particular, this implies that decoding would fail even if 
we choose extremely large variable degrees. For example we 
might consider a sequence of codes defined by parity-check 
matrices that contain all rows of a given weight u; row . Clearly, 
a specific code C n of this sequence is a w n ~ ^ 1 J, w mw )- 
regular code which contains either one (C n = {0}) or two 
codewords (C n — {0, 1}) depending on k being odd or even. 
Thus, while the rate of this code sequence approaches zero, 
LP decoding will not succeed for e > □ 

1-1 1—1 '"■ow 

Example 10: Th. |6| and Cor. |7J can easily be extended to 
families of codes where the row weight grows as a function 
of n; let us call theses extensions Th. |6] and Cor. . It is clear 
from Cor. that there cannot be an LP decoding threshold 
for the BSC if the row weight grows unboundedly. Moreover, 
coming back to the code family in Ex. |9] if w row is allowed 
to grow with n, LP decoding will fail as n — > 00 despite 
the variable degree being exponentially larger than the check 
degree. 

Example 11: Similarly, the family of random codes where 
entries in a parity-check matrix are drawn independently from 



Fig. 2. The solid line shows the capacity Cbsc (s) of a BSC as a function 
of the cross-over probability e. The circles have the following meaning: the 
circle with label q shows the point (R(q), £tjb(9)) > where R(q) is the rate 
of the PG(2, q)-based code and where £ub(<?) is the threshold upper bound 
from Cor. 0for a family of (ui co i(g), «i r0 w(9))-regular LDPC codes. (See 
also the main text for further explanations.) 
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a Bernoulli^) distribution will not only have poor threshold 
performance under LP decoding but will fail with high proba- 
bility as the code length approaches infinity for any symmetric 
channel for which the expression 

E[r | r>o] Pr(r>o) 
~E[r|r<o] ' Pr(r<o) 

is (upper) bounded. The result follows from the observation 
that the weight of the rows in H is exponentially concentrated 
around On. Indeed, given a vector of log-likelihood ratios, the 
vector with components ng } s _ 1 in positions where 7* is non- 
negative and 1 in the remaining positions is inside /C(H) with 
high probability for 5 > and n — > 00. □ 

While the above considerations give some insight in the 
asymptotic behavior of of decoding error for LP decoding, the 
characterization and spirit of Th.[6]is essentially combinatorial. 

Example 12: We saw in Ex.^|that for any family of codes 
where the row weight grows as a function of n, Cor. [71 implies 
that there cannot be an LP decoding threshold for the BSC. A 
special case, though, arises when the rate of the code family 
under consideration goes to 1 when n — > 00 because then 
also the best code family under the best possible decoding 
algorithm can only correct a vanishing fraction of bit flips as 
n — ► 00. A family were the rate goes to 1 as n — ► 00 is the 
family of type-I PG(2, g)-based codes, cf. [13]; in the context 
of LP decoding, these codes were analyzed in [14], [15]. A 
code from this family is indexed by q (where q = 2 s for some 
positive integer s), has length n(q) = q 2 + q + 1, rate R(q) = 
1 - (3 s + l)/(q 2 +9 + 1), and w co i(q) = w mw (q) = q + 1. 

Fig. 13 shows the following: for each q we plot the point 
(R(q), £ub (<?)), where R(q) is the rate of the PG(2, </)-based 
code and where £tjb(<z) is the LP decoding threshold upper 
bound from Cor. [7] for a (w co \(q), w IOW (q)) -regular family of 
codes. Care must be taken when giving an interpretation to this 
figure since the PG(2, q)-based codes are finite-length codes 



Fig. 3. The solid line shows the capacity Cbsc (s) of a BSC as a function of 
the cross-over probability e. The circles have the same meaning as in Fig.fTI 
The squares have the following meaning: the square with label q shows the 
point £tjb(<?)) ■ where R(q) is the rate of the PG(2, q)-based code 

and where e\jb (q) is the 2-neighborhood-based threshold upper bound from 
Sec. II VI for a family of («> co i(<?), tu row (q))-regular LDPC codes. (See also 
the main text for further explanations.) 



for finite q. □ 
We leave it as an exercise for the reader to generalize the 
results in this section to irregular LDPC codes. 

IV. 2-Neighborhood-Based Bounds on the 
Threshold for Regular LDPC Codes 

Because the assignment of a value to u>{ in Q was only 
based on the value of 7$, we call the resulting bound in Cor. 0a 
O-neighborhood-based bound. (Of course, the way we assigned 
a value to every u>i in 10 can also be seen as a very simplistic, 
and usually sub-optimal way, of solving the linear program 
in Q.) It is natural to try to formulate more sophisticated 
assignments of a value to u>i. The next simplest approach 
is to formulate a rule that does not depend on 7; only, but 
also on 7j' where i' ranges over all variable nodes at Tanner 
graph distance 2 from variable node i. The resulting bounds 
on the threshold will therefore be called 2-neighborhood-based 
bounds. 

For i G X let J\Q be the subset of 2 that includes 
all variable nodes i' with Tanner graph distance at most 2 
from i. In the following, we assume that the Tanner graph 
under consideration has girth at least six. In the case of a 
(w C oi, w r ow)-regular LDPC codes, this implies that the set 
A/f) has size |A^ (2) | = 1 + w col {w mvi - 1). (Fig. H (left) 
shows part of a (3, 4)-regular code, i.e. node i, all check nodes 
at Tanner graph distance 1 from i, and all variable nodes at 
Tanner graph distance 2 from i.) 




Fig. 4. Left: 2-neighborhood. Right: overlapping 2-neighborhoods. (See 
Sec.lTvlfor more explanations.) 



Definition 13: Let be the vector that contains all the 
random variables {IV}j, 6A/ -(2) and let 7? be its realization. 
Our new rule (that replaces (|5Jl) for defining a vector u> £ M" 
is now LOi = a~(, where ay G M+ for all 7? e 5^ I is 
chosen such that for all possible 7 e CT 1 we obtain a vector 
u> that lies in the fundamental cone. 

Lemma 14: Assume that we have such values {a^'}^' as 
defined in Def. ^] With probability one with have 

lim - V 7^ = V p-y'.a^ , 

iex -y'. 

where 

p yl Ap r ( { r i ,}. /eA/ . (2) ={7 i ,}., £Ar(2) )= J] Pr(T = 7 iO- 

*'eA/f> 

Proof: Follows from the fact that jiUii depends only on finitely 
many 7^ from {7i'}i'ez an d from the use of the weak law of 
large numbers. □ 
Consider a BSC with cross-over probability e. An upper 
bound on the LP decoding threshold for the BSC is then given 
by the infimum of all e were we are able to find an assignment 
in Def. [O] such that lim.,^00 — J^iei "f iU>i * s negative with 
probability one. Finding such assignments can e.g. be done 
by solving a linear program that roughly looks as follows 

min. y"lp-r< a *r< 

subj. to ay = 1, 

and for each 7 the assignment always results in a 
non-zero vector that lies in the fundamental cone, 

where 7' is an arbitrary assignment of values to 7^, e.g. 7^ = 
— G for all i' £ Af t , where G is defined as in Ex. |2] 

The Tanner graph in Fig. |4] (left) has many symmetries that 
can be used to simplify the above linear program. E.g. if there 
is a graph isomorphism that maps an assignments 7^ to an 
assignment 7^, then without loss of generality we can assume 
that cby = a=> . In this way, the dimensionality of the above 
linear program can be reduced significantly. Without going 
into the details, the requirement that the resulting assignment 
always results in a vector in the fundamental cone can be 



simplified by introducing some auxiliary variables according 
to overlapping 2-neighborhoods as is sketched in Fig. |4] (right). 

Example 15: Fig. [3] shows the improved upper bounds for 
(w co i, w row )-regular code families. □ 

Whereas the above approach results in relatively small linear 
programs for small w co i and w mw , similar 4-, 6-, 8-, etc., 
neighborhood-based approaches seem to be computationally 
much more demanding. We leave it as an open question to 
see if there are ways to handle also these cases in an efficient 
numerical way. 
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